Hi @DennisOSRM and @TheMarex please review my PR -

This PR should not be merged because:

• it tests only 3 build variants, and 1 OS, whereas all variants should be tested,
• while testing it still uses only 1 cpu per runner and there is a limit of 20 concurrent runners per repo, so it is using only 20 cpus (out of ~80 available)
• testing will have to wait until the longest build is complete, meanwhile runners are idling, and it fails later,
• it changes nothing for the developer at home, which still has to put up with terrible running times.

There is the better previous PR #7309 which:

• will test all builds variants if configured to do so (linux, macOS and Windows) ,
• uses all available CPUs,
• starts testing early and fails fast,
• delivers even more speed gains at home (total run time about 30s with 16 cpus).

Hi @MarcelloPerathoner this PR is not intended to replace your work in #7309.

The two pull requests are actually complementary: my PR parallelizes across configurations (different build configs, algorithms, load methods) using GitHub Actions matrix, while yours parallelizes within a single configuration using cucumber's --parallel flag.

In theory both could be combined - the matrix creates 18 jobs, and each could benefit from your --parallel optimization internally. But if the maintainers prefer to wait for #7309, I'm fine with closing this one.

Hi @MarcelloPerathoner this PR is not intended to replace your work in #7309.

The two pull requests are actually complementary: my PR parallelizes across configurations (different build configs, algorithms, load methods) using GitHub Actions matrix, while yours parallelizes within a single configuration using cucumber's --parallel flag.
In theory both could be combined - the matrix creates 18 jobs, and each could benefit from your --parallel optimization internally. But if the maintainers prefer to wait for #7309, I'm fine with closing this one.

Combining #7309 with this PR would only make things slower (and messier).

Take this data from an actual debug run and do the math:

After the debug build is done:

• Parallel cucumbers #7309 will run those 6 tests on 4 CPUs. Total time (sum of the above): ~ 270s.
• this PR will setup 6 new runners. The slowest test above will run about 4 times slower on a single CPU. Total time ~ 4*117s = 468s.

And that only if it can get a runner immediately. If you test more than 3 variants you are guaranteed not to get a runner immediately. If you test 15 variants as we currently do, you'll need 90 runners.

P.S. #7309 is ready for review.

• ubuntu-latest runners have 2-4 vCPUs, not 1. The "4x slower on a single CPU" claim is incorrect.
• With the matrix approach, all 6 configurations can run in parallel, so wall-clock time would be ~117s (the slowest config), not 468s
• To combine both PRs, merge Parallel cucumbers #7309 first (for port/thread-safety), then change the cucumber-tests job to:

- name: Run cucumber tests
  run: OSRM_BUILD_DIR=... npx cucumber-js features/ -p ${{ matrix.algorithm }} --parallel 2

It runs faster only because you are testing 3 variants instead of 15. If you test more than 3 you will starve of runners and nothing is gained. Try testing 15 variants and report your times and we will see.